G 3 . 5 Learning to break things : adaptive testing of intelligent controllersAlan

Autonomous vehicles require sophisticated software controllers to maintain vehicle performance in the presence of vehicle faults. The test and evaluation of complex software controllers is a challenging task. The goal of this eeort is to apply machine learning techniques to the general problem of evaluating an intelligent controller for an autonomous vehicle. The approach involves subjecting a controller to an adaptively chosen set of fault scenarios within a vehicle simulator, and searching for combinations of faults that produce noteworthy performance by the vehicle controller. The search employs a genetic algorithm. The evidence suggests that this approach is an eeective supplement to manual and other forms of automated testing of sophisticated software controllers. Several intelligent controllers were tested in this project using several diierent genetic algorithm-based learning programs. Over the course of this research, the representation, evaluation function, genetic operators and the basic algorithm themselves evolved. This section presents this work from the point of view of describing the process that the authors followed in applying these learning algorithms to this real-world problem. Autonomous vehicles require sophisticated software controllers to maintain vehicle performance in the presence of faults. The test and evaluation of such a software controller is a challenging task, given both the complexity of the software system and the richness of the test environment. The goal of this eeort is to apply machine learning techniques to the general problem of evaluating a controller for an autonomous vehicle. The approach involves subjecting a controller to an adaptively chosen set of fault scenarios within a vehicle simulator, and searching for combinations of faults that produce noteworthy performance by the vehicle controller. The search employs a genetic algorithm (GA), i.e., an algorithm that simulates the dynamics of population genetics (Holland 1975, De Jong 1980, Grefenstette et al 1990, Schultz and Grefenstette 1992), to evolve sets of test cases for the vehicle controller. We have illustrated the approach by evaluating the performance of two diierent intelligent controllers, one for an autonomous aircraft and the other for an autonomous underwater vehicle. The evidence suggests that this approach ooers advantages over other forms of automated and manual testing of sophisticated software controllers, although this technique should supplement, not replace, other forms of software validation. This research is signiicant because it provides new techniques for the evaluation of complex software systems, and for the identiication of classes of vehicle faults that are most likely to impact …